1. Field of the Invention
The invention relates to a device and a method for the processing of remote sensing data. The invention relates in particular to remote sensing image data of objects on the earth's surface and/or the atmosphere, where the image data is obtained from remote sensing satellites, aircraft or other flying objects (e.g. balloons).
2. Description of Related Art
In past years satellite systems have been additionally equipped with sensor packages, where several initial sensors with a local coarse resolution (because of the differently designated spectral ranges also called multi-spectral sensors) were extended with a second local fine resolution, panchromatic sensor, where the multi-spectral sensors are more sensitive to incident radiation in a narrower spectral range in each case (wavelength range) than the panchromatic sensor. On the one hand, this trend follows from the fact that many applications require a high spectral resolution for satellite data with a fine local resolution, on the other hand, however, the transfer quantity in case of transfer of remote sensing data from a satellite to the ground station is limited. In particular in the areas of geology, the land utilization mapping and/or the updating of maps, e.g. for the agricultural and forest economy, the data for applications are used, which are subsumed into the term “change detection”. Another region relates to the monitoring of natural catastrophes.
Typically the data (image data) of the multi-spectral bands in the remote sensing are used for the derivation of the thematic information (e.g. the properties of the reflecting surface), where the data of the panchromatic channel are used for the extraction of spatial information. The objective is also to have available for further processing the thematic information with maximum high resolution at least corresponding to the local resolution of the panchromatic data. The panchromatic image data can be employed for this refinement of the local resolution of the multi-spectral data since, they were generally recorded at the same time and represent the same remote sensing objects.
With the measuring data of the multi-spectral bands and the panchromatic band and/or the channels, it involves digital image data. The remote sensing images are nowadays mostly digital measuring data of radiation-measuring sensors in different spectral ranges, with which the images are two-dimensional. The arrangement of the image elements (pixels) is also designated as image matrix or image array. Every single measurement of a sensor element of the image-providing device produces a section or a small, almost point-like section of the image. The section is generally regarded as quadratic, so that the edge length of the section, which is also designated as a resolution, can be calculated from the square root of the section size. However, the sections examined can also indicate a different edge length in the different spectral ranges.
The single measurements which must be implemented for the formation of an image matrix can be implemented either in series (which means one after each other) or in parallel (which means simultaneously). The type of origin of the image matrices is of subordinate importance for the process of refinement of the local resolution. Furthermore, not only one image matrix is usually recorded but image matrices are continuously recorded. Preferably the image data of the multi-spectral channels are therefore refined for every recording time point or every image matrix.
The trend is noted that the sections to be measured become increasingly smaller, i.e. the spatial resolution becomes increasingly better such that the entire surface areas examined become increasingly larger, i.e. the total number of sections (pixels) become ever greater and the number of examined spectral ranges also increases. The refinement of the local resolution of the multi-spectral channels targeted in this case determines an exponential rise of the data quantity to be processed. In many cases, it is required that the data be processed with the speed (rate) with which it is recorded (i.e. digitally stored). Processing in real time is referred to in this case.
A useful application of the refined multi-spectral image data is e.g. disaster management, for example in the case of forest fires on a large surface area, volcanoes, tsunamis and earthquakes. In this case, the processing of fine-resolution, multi-spectral data represents a significant advantage compared to the utilization of low-resolution data.
The digital image data must not necessarily be obtained directly from digital cameras, as designated above. Rather it is also possible to use other remote sensing processes, at least for a part of the multi-spectral channels, such as e.g. radar systems with the possibility of the generation of locally fine-resolution data which, however, also in case of cameras, can be stored in the form of digital image data with pixels arranged in lines and columns. In the case of satellites, the fine and coarse resolution digital image data do not necessarily have to be obtained from the sensors of only one satellite. For example, a sensor of a further satellite and/or a radar system can be used in addition.
Different processes for the refinement of the resolution of multi-spectral digital image data are known. In this case, it involves statistical methods and methods which are based on color space transformations, or of methods which are based on physical considerations. The objective of all refinement methods is to predict measured values (pixels) at image locations, which have actually not been measured. From the multi-spectral channel there are only pixels present which correspond to a large local area in each case. Furthermore, the local finer resolution image information from the panchromatic channel is present only with a smaller spectral resolution. The result of the refinement of the multi-spectral image data can therefore be correct with a certain probability only, however, not with ultimate certainty.
For example, from EP 1 626 256 A1, a method for the processing of multi-spectral remote sensing data is known. In a local area or section of the panchromatic measuring data, a local distribution of a radiance of a reflection factor recorded in the panchromatic measuring data, or an equivalent radiation extent is determined and, considering a balance of at least a part of the radiation energy recorded through the measuring data or a balance of the radiation energy equivalent size, is transmitted to a corresponding local area or section of the multi-spectral measuring data, so that the local resolution of the multi-spectral measuring data is refined. In spite of the consideration of the balance of the radiation energy, the result of the refinement with this process is also correct with a certain probability only, i.e. it corresponds to the actual conditions in case of directly-refined measurement with the certain probability only.
The different known processes are differentiated not only through the process principles which underlie them, but also through different resource requirements on main memory and processor performance of a data processor or data processor system. In addition, there are processes which can be virtually realized better or worse through digital data processors.
Since, with the image data, it involves digital image data, the data processing time required for the refinement is a significant criterion as to whether or not the process is suited for the appropriation of the refined data in real time.
Some known processes use so-called wavelet transformations for the refinement, the method of main-component analysis or other transformations which lead to inconsistent, non-color-retention results. Other known processes require so-called classified image data for the refinement of the multi-spectral data. This requires an extensive preprocessing of data with additional data for classification.
The publication of J. G. Liu “Smoothing Filter-based Intensity Modulation: a spectral preserve image fusion technique for improving spatial details”, published in Int. J. Remote Sensing, 2000, Vol. 21, No. 18, Pages 3461-3472 describes a data fusion technology to spatially integrate lower-resolution, multi-spectral images with spatially higher-resolution panchromatic images. A Pixel Block Intensity Modulation (PBIM) technology is further developed to a Smoothing Filter-based Intensity Modulation (SFIM) technique. For every pixel of the high-resolution image, a local average value is calculated with a smoothing fold filter. The filter core size is decided based on the resolution ratio between the higher and lower resolution images.
In order to enable the finer and coarser resolution image data to be fused with each other, the data should be co-registered as far as possible, i.e. it should be exactly known over the entire recorded image field which location in one image corresponds to which location in the other image. Since an exact co-registration frequently does not succeed in practice, however, Liu proposes in his publication to select the size of the filter core by means of a statistical correlation analysis of the different images. A smearing of edges not matching completely with each other is reduced through enlargement of the filter core, i.e. the resulting image is sharpened at the edges of structures represented. On the other hand, according to Liu, a too large core size leads to an edge improvement which is not absolutely desired. The edges become apparently sharper, however, because of the strong smoothing effect, no longer correspond to reality. In particular, the histogram function of the initial, non-refined image is modified by the smoothing in the refined resolution image. In local areas, a change of the integral of the image values over the local area occurs through the smoothing.
In addition, the application of a smoothing filter increases the computation time for the data fusion. This is contrary to the requirement that the data is refined with regard to its resolution, as far as possible in real time at the location of its extraction (e.g. in a satellite), and is transmitted to a remote receiving station, in particular on the planetary surface, or the resolution improvement is implemented in real time immediately with the data reception on the planetary surface. If further, new image data result through the on-going observation of the planetary surface than are capable of being refined, a transfer in real time is no longer possible.